In many data processing applications, including those used in networking and communication applications, input data is provided to the processing system in a number of packetized data streams. Packets that belong to the same stream are typically processed using the same ancillary data, sometimes referred to as commands and context, such as state information for the packets, operands from a memory, and other data. Example ancillary data can include security keys or tokens, for use in processing streams associated with encrypting and/or decrypting data, and other metadata used to process data packets. Often, the data processing operations are offloaded from the main processing architecture to specialized hardware that performs the various data processing operations. Systems often limit the number of packets from the same stream that can be in a prioritization queue/prefetch module at the same time to allow packets from other streams to reside in the queue contemporaneously.